3D Subject-Atlas Image Registration for Micro-Computed Tomography Based Characterization of Drug Delivery in the Murine Cochlea

A wide variety of hearing problems can potentially be treated with local drug delivery systems capable of delivering drugs directly to the cochlea over extended periods of time. Developing and testing such systems requires accurate quantification of drug concentration over time. A variety of techniques have been proposed for both direct and indirect measurement of drug pharmacokinetics; direct techniques are invasive, whereas many indirect techniques are imprecise because they rely on assumptions about the relationship between physiological response and drug concentrations. One indirect technique, however, is capable of quantifying drug pharmacokinetics very precisely: Micro-Computed tomography (micro-CT) can provide a non-invasive way to measure the concentration of a contrast agent in the cochlea over time. In this thesis, we propose a systematic approach for analyzing micro-CT images to measure concentrations of the contrast agent ioversol in mouse cochlea. This approach requires segmenting and classifying the intra-cochlea structures from micro-CT images, which is done via 3D atlas-subject registration to a published atlas of the mouse cochlea. Labels of each intra-cochlear structure in the atlas are propagated through the registration transformation to the corresponding structures in the micro-CT images. Pixel intensities are extracted from three key intra-cochlea structures: scala tympani (ST), scala vestibuli (SV), scala media (SM) in the micro-CT images, and these intensities are mapped into concentrations using a linear model between solution concentration and image intensity that is determined in a previous calibration step. To localize this analysis, the ST, SV, SM are divided into several discrete components, and the concentrations are estimated in each component using a weighted average with weights determined by solving a nonhomogeneous Poisson equation with Dirichlet boundary conditions on the component boundaries. We illustrate this entire system on a series of micro-CT images of an anesthetized mouse that include a baseline scan (with no contrast agent) and a series of scans after injection of ioversol into the cochlea

Specification tests for temporal heterogeneity in spatial panel data models with fixed effects

Singapore Management Universit

Towards Deep Visual Learning in the Wild: Data-efficiency, Robustness and Generalization

Deep Learning approaches have achieved revolutionary performance improvement on many computer vision tasks from understanding natural images and videos to analyzing medical images. Besides building more complex deep neural networks (DNNs) and collecting giant annotated datasets to obtain performance gains, more attention has now been focused on the shortcomings of DNNs. As recent research has shown, even when trained on millions of labeled samples, deep neural networks may still lack robustness to domain shift, small perturbations, and adversarial examples. On the other hand, in many real-world scenarios, e.g. in clinical applications, the number of labeled training samples is significantly smaller than for large existing deep learning benchmarks. Moreover, current deep learning models cannot generalize to samples with novel combinations of seen elementary concepts. Therefore, in this thesis, I focus on handling the critical needs to make modern deep learning approaches applicable in the real-world with a focus on computer vision tasks. Specifically, I focus on data efficiency, robustness, and generalization. I propose (1) DeepAtlas, a joint learning framework for image registration and segmentation that can learn DNNs for both tasks from unlabeled images and a few labeled images. (2) RandConv, a data augmentation technique that applies a random convolution layer on images during training to improve the generalization performance of a DNN in the presence of domain shift and robustness to image corruptions. (3) CompGen, a comprehensive study of compositional generalization in unsupervised representation learning on disentanglement and emergent language models.Doctor of Philosoph

SimpleClick: Interactive Image Segmentation with Simple Vision Transformers

Click-based interactive image segmentation aims at extracting objects with a limited user clicking. A hierarchical backbone is the de-facto architecture for current methods. Recently, the plain, non-hierarchical Vision Transformer (ViT) has emerged as a competitive backbone for dense prediction tasks. This design allows the original ViT to be a foundation model that can be finetuned for downstream tasks without redesigning a hierarchical backbone for pretraining. Although this design is simple and has been proven effective, it has not yet been explored for interactive image segmentation. To fill this gap, we propose SimpleClick, the first interactive segmentation method that leverages a plain backbone. Based on the plain backbone, we introduce a symmetric patch embedding layer that encodes clicks into the backbone with minor modifications to the backbone itself. With the plain backbone pretrained as a masked autoencoder (MAE), SimpleClick achieves state-of-the-art performance. Remarkably, our method achieves 4.15 NoC@90 on SBD, improving 21.8% over the previous best result. Extensive evaluation on medical images demonstrates the generalizability of our method. We further develop an extremely tiny ViT backbone for SimpleClick and provide a detailed computational analysis, highlighting its suitability as a practical annotation tool.Comment: Tech report. Update 03/11/2023: Add results on a tiny model and append supplementary material